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METHOD AND APPARATUS FOR DATA COMMUWICATION BETWEEN A 

PLURALITY OF PARTIES 



5 FIELD OF THE INVENTION: 

The presmt invention relates to a system and method of providing secure commmiications over 
an open network, and more specifically to establishing a virtual private network (VPN), which 
runs across a diveise set of operating systmis and hardware platforms and &cilitates ease of 
10 use. 

BACKGROUND: 

Workgroup computing involves, by definition, the exchange of data between the nodes of the 
15 workgroup, a node being a computer connected to a net\york which can be identified with an 
individual, a set of resources (files, services, devices, etc), or a gateway. Often, the tasks of a 
workgroup are of a sensitive nature containing, for instance, confidential data on finances, 
business development plans, or private email. The Internet (and its^ native IP protocol) has 
become ubiquitous as a means of connecting nodes in a workgroup computing environment. 
20 However, with the adoption of the Intemet and its public networking infrastructure comes the 
risk that an unauthorised 3"^ party with access to the data route between two nodes may 
intercept and reconstruct data transferred between them. To prevent interception, a mechanism 
is required to modi^ the transmission of data such that only the intended receiver may interpret 
it and the receiver can be guaranteed of the data origin and integrity. 

25 

A virtual private network is a logical entity consisting of multiple nodes havmg a secure 
communications over an open and typically msecure network such as the Intemet. Data 
security is commonly achieved through the use of cryptography, which requires the data 
traffic to be encrypted at the sender's end and then decrypted at the receiver's end so that 
30 other users of the public network can intercept the data traffic, but cannot read it due to the 
encryption. Data encryption also allows the receiver to* verify the integrity of the data 
received and therefore detect 3"* party data tampering. 
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A typical VPN connects one or more private networks together through the Internet. 
Generally, the network on either side of the Internet has a gateway and a single-access 
connection to the Internet To create the VPN, a secure communications path between the two 
gateways is formed such that the two private networks may communicate with one another. 

5 

In order to establish secure communication between any two nodes on a VPN, each node 
obtains by some means information ("configuration*') including but not limited to: 

• The identity and state ofthe remote nodes within tiie VPN 

• The relationships between nodes (VPN topology) 

10 • Cryptography for authentication and data conmumications encryption between nodes,* for 
example the key for a VPN based on shared secrets or certified public key for VPN 
utilizing Public Key Lifirastructure (PE^. 

Secured communication between two nodes is commonly called a 'tunnel*, while nodes 
15 themselves are often referred to as 'tunnel terminators'. Traditional VPN solutions are 
comprised of a number of tuimel temiination devices, which provide a central "hub*' for VPN 
communication. Software is then deployed to nodes that wish to participate in a VPN, and tiie 
software is configured manually with the address of the VPN deviceOs). The software is tiien 
executed in order to participate in the VPN. However, there are several disadvantages with 
20 respect to this technology. In general, a VPN does not allow for automatic configuration of 
nodes for VPN participation as nodes change tiieir network addresses on being dynamically 
added/removed to/firom a VPN. In addition, each of tiie nodes may only be a member of one 
VPN at a time in the majority of inaplementations, which limits the ultimate efficiency of the 
user at each node 

25 

The use of VPN*s is well known in the computer world each using different mechanisms to 
provide a means of secure data transmission. United States Patent No. 6,061,796 entitled 
"Multi-Access" Virtual Private Network describes system and method for allowing private 
communication over an open network. This system however, specifies what mechanism 
30 protocol level the Agent (VPN provisioning application) uses to intercept incoming and 
outgoing data firom a node and is not designed to work with IP networks. In addition, it 
would be difficult to scale this particular system for large-scale use. In United States Patents 
No. 5,884,035 and 6,026,430 data transmission is only through tiie domain hierarchy and not 
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on a data to client application basis. In the VPN system described in United States Patent 
No. 6,055,575 it notes that the 'Tiost computer estabUshes a secure communications path, 
referred to as a tunnel, through the public network with the remote clienf \ This has firewall 
implications in that a remote node can rarely accept incoming connections. 

Another very common limitation of traditional VPNs is their inability to cross boundaries of 
private networks linked to each other through one or more Network Address Translation 
(NAT) devices. Jn addition, existing VPN do not facilitate the use of end-to-end security m 
the presence of firewalls, gateways, and proxy servers. NAT devices, botti regular and PAT 
are very widely deployed to allow for better security by hiding details of private network 
&om the outside world and to facilitate conservative use of public IP addresses by mapping 
multiple private addresses onto single public one. With the growfli of the Internet and delayed 
introduction of version 6 of IP protocol (Ipv6), more and more companies will be forced to 
use NAT devices as IP address space available for gmeral public becomes increasingly 
exhausted. The above-mentioned limitation arises because a NAT device modifies the data 
packet to allow for proper routing both inside a private LAN^.and in the outside world. 
However, any change to the packet is treated by tunnel terminators as a tampering, thus 
packets xmdergoing NAT processing are discarded as damaged 

As it follows fi-om known PAT functioning principles, the presence of post-EP header is a 
necessary condition for the packet to be translated by the PAT. Also, since a PAT device maps 
all internal nodes onto a single IP address, it creates and maintains intemal associations 
between IP address and post-IP header of the intemal node and its translated post-IP header. 
This means that traffic traversing PAT device and destined for an intemal node requires a 
proper association to be in place to fiacilitate the reverse moping. lii other words, any post-IP 
session between PAT'ed and external node may only be initiated by the external node. 

It is an object of the present invention to obviate and mitigate at least some the 
aforementioned disadvantages of the prior art. 
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SUMMARY OF THE INVENTION: 



Accordingly one aspect of the present invention provides a system for facilitating the secure 
commxinication between nodes in a workgroup by the creation of an "n"-tiered virtual private 
5 networic (VPN). Each node preferably has the ability to transmit and receive secured data 
ova: a pubUc network such as the Internet The system comprises at least a pair of nodes, a 
server, a datastore linked to the server (where the datastore may be in the form of memory, a 
disk, a database etc), and a client application capable of communicating with the VPN server 
and securing P-level connections towards other VPN nodes by utilizmg a suite of protocols, 

10 for example and IPSec protocol, in particular an ESP protocol The datastore furth^ includes 
information pertaining to the configuration of VPNs, VPN relationships (e.g. client computer 
membership to VPN's), settings and options (e.g. IPSec ciphers to use), authentication 
information, and objects and attributes (e.g. status - online/ofOine, human-readable node 
description, node IP). The system further includes a means to intercept both incoming and 

IS outgoing data from a node so as to create a secure tunnel between an open network and a 
node by encrypting and decrypting data. In addition, flie system includes a means for 
verification of node credentials against authentication servers, the tunnel enables data to be 
securely shared to VPN(s). 

20 The present invention is designed to faciUtate the aspects of VPN fimctionaHty including but 
not limited to: securing communication within the VPN and VPN configuration for the 
exchange of secure infonnation between VPN nodes. 

In another embodiment, on start up of a node within the system, the client forms a connection 
25 with the VPN server. Authentication credentials are transmitted to the VPN server, where 
they are vahdated and a connection is established. Following the creation of a secure 
cormection between the VPN server and a node, the client application is synchronized with 
the VPN server by receiving and processing initial configuration infonnation. This 
information includes a list of VPN's of which this particular node is a member, their 
30 respective attributes, a listing of other nodes which are members of the same VPNs as the 
client computer, the current status of each node in each respective VPN, and other related 
details. Once a node is logged onto and synchronized with the VPN server its client 
application sits in the loop so as to maintain the node in sync with the rest of the VPN by 
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sending and receiving status and configuration updates to/fi-om VPN server. The central 
management of the system enables the server to be informed of any changes to a VPN e.g. a 
node logging oS, and is informed of these changes in a timely manner, where the time fi-ame 
is elected by the node. The VPN server then relays this information to each node within the 
5 VPN, which in turn is putting its sel£ the VPN server, in sync with the system. 

This system is global by the nature of the server such that it fecilitates the central 
management of any VPN. The server fedUtates the ability to make changes to a VPN 
without having to effect changes manually at each node of a virtual private network. A 

10 change made to the datastore linked to the server is transmitted in a timely manner To all 
client computers effected by this change. For example, to change the password of a VPN for 
each node in a network reqmres making that change to the datastore and, m tum, that change 
is transmitted to each node on the virtual private network. While changing a password is a 
relatively sisjople task, the ability to effect more detailed changes to a VPN requires updatmg 

15 only a smgle point in a VPN and then transmitting that data to the remaining nodes in the 
workgroup via the secure connection. In use, the network includes the ability to automatically 
and securely provision security associations between nodes. 

The control of the VPN created using the VPN server may be in house in the sense that, at a 
20 particular company subscribing to this service, an DP manager would administer and maintain 
the VPN and have rights to modify information on the server and datastore as it pertains to 
their VPN. Generally, IP traffic between two nodes on a VPN is encrypted and decrypted 
regardless of the type of information being sent. The decision as to secure the channel 
between two nodes or not is made by VPN server based on the topology configuration of the 
25 VPN. The server itself however, does not participate in node-to-node data transfer. 

This invention fiuiher provides a system to enable secure communication between nodes over 
the Internet and have the benefit of end to end security. This system enables a node, which 
may operate behind generic NAT box and/or a firewall, to establish and use secure 
30 communication over the Internet with another node. In general, there are two different types 
of Network Address Translation (NAT) devices - regular NAT and Network Port Address 
Translation. The difference between these two types is that a regular NAT device uses IP 
header information to relay packets to and from members of a private group. Network 
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Address Port Translation uses an IP and transport layer protocol (TCP/UDP/ICMP) header. 
This is also referred to as PAT. 

The system comprises at least a pair of nodes belonging to the same virtual private network, a 
5 packet interception mechanism, a secure line for communication to the VPN server, and a 
cUent application located at each node. The client application located at each node includes a 
mechanism to enciypt, decrypt or process data exchanged within the virtual private network, 
and a software module responsible for maintflming configuration information including VPN 
relationships, authentication information, and settings and options. In addition, the 
10 configuration information mdicates the presence of a NAT device, firewall, gatewayrand 
proxy server in firont of particular nodes m a VPN. The system further comprises a 
mechanism for verification of liode credentials against authentication servers, which enables 
data to be . securely shared amongst members of a private group. The packet interception 
mechanism is generic and known to one skilled in the art. 

15 

Once nodes are logged onto a VPN, they may exchange information. Outgoing data packets 
are intercepted and then those destined to a specific VPN node are selected for further 
processing. When ongoing data packets are intercepted, the VPN indicates the presence of a 
NAT or PAT device, a firewall, gateway, and proxy server in fix>nt^f the mtended receiving 

20 node. In order to facilitate data exchange to nodes located behind one of the above- 
mentioned devices, the data packet header is modified. The data packet itself is encrypted as 
a whole and a. new header is prepended to the now encrypted data packet. Source and 
destination node information is added to the prepended header and is determined by the VPN. 
The new header is referred to as an "external header" and the origmal packet header is 

25 referred to as tiie "internal header". The external header contains a masquerade bit which 
allows the receiving node to recognize the modified data packet as having a prepended 
external header. Once the data packet traverses the device, the external header is removed 
and the packet is processed according to the specifics indicated by the original IP header. 

30 BRIEF DESCRIPTION OF THE DRAWINGS 
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These and other features of the preferred embodiments of the invention will become more 
apparent in the following detailed description in which reference is made to the appended 
drawings wherein: 

5 Figure I : is a schematic diagram of an overview of a computer system; 

Figure 2: is a functional block diagram detailing the method for establishing secure 
communication 
betwera nodes, in the computer: system of figure 1; 
Figure 3: is a schematic of the computer system incoipoxating a plurality of types of nodes; 
10 Figure 4: is a schematic diagram of an overview of a computer system incorporating LAN's, a 
gateway, and a firewall; 
Figure S: is a functional block diagram detailing ttie method for sending-data ov^ a VPN 
having 

secure communication in the conq>uter system of figure 1; 
1 5 Figure 6: is a jSmctional block diagram detailing the method for receiving data over a VPN 
having 

secure commvmication in the computer system of figure 1 ; 
Figure 7: is a schematic of the data packets transferred between a pluraUty of types of nodes on 
a VPN; and 

20 Figure 8: is a schematic diagram of an overview of another embodiment of the computer 
system of Figure 1. 



To facilitate the understanding of the preferred embodiments described below, the following 
25 terminology will be used, it being understood that this is for illustrative purposes only and is 
not limiting: 

Client Application - the software that acts as a slave to a server and is present on each 
node within a work groiq); 
30 VPN - a virtual private network that is constructed over a public network to connect 

nodes 

within a work group such that: 
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a) data transfeired between those nodes is secure and cannot be 
intercqjted, modified, or replaced on route; and 

b) it contains mechanisms to ensure that only authorized users may access 
the network, 

5 Node - a computer connected to a network which maybe identified with an individual, a 

set of resources, or gateway; 

Work Groiq).- a group of two or more individual nodes working collaboratively on a 
group of tasks; 

Gateway - a special node that provides secure communication to a specific network of 
10 nodes located behind the gateway; and 

Network Address Translation - (NAT) an Intemet Standard that enables a LAN to use 
one set of IF addresses for internal trafSc and a second set of addresses for external 
trafi&c. 

15 DETAILED DESCRIPTION OF THE EMBODIMENT 

A system and method for establishing a secure connection for th&itransfer of data between 
nodes in a work group over a public network is illustrated in figures 1 through 8. The 
conq)Uter system is generally designated by reference numeral 10. The system 10 may be 
20 configured in a numb^ of different ways including those utilizing individual users as shown 
in Figure 1, those utilizing individuals and intranet as shown in Figure 3, and those utilizing a 
gateway as shown in Figure 4. Initially it is necessary to establish conmiunication between 
memb^ of virtual private network (VPN) and this procedure will be described in respect of 
each configuratioh. 

25 

As shown in Figure 1, a computer system 10 comprises a plurality of nodes 12 (client 

computers), server 18, and a datastore 20 whose contents may be updated or changed 

periodically by external intervention. Server 18 is also referred to as the VPN server however, 

it is understood that the VPN server is cq)able of performing typical server fimctions known 

30 in the art in addition to the provisioning of a VPN as is described below. Each of the nodes 

12 includes a client application 14 capable of communicating with server 18. The system 10 

is arranged to enable the establishment of a secure path for commimication between nodes 12 

over a public network such as the Internet 22. The server 18 collects and distributes data 

collected by the client application 14 at each node 12, so as to maintain state information for 
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each node 12. The server 18 tracks changes made to the datastore 20 and subsequently 
updates each of the nodes 12. The client appUcation 14 is responsible for transmitting 
information to and receiving information from a second cUent appUcation 14 of a node 12 
and server 18. The server 18 also serves to generate specific node cues based on those 
5 events, such as the availability of upgrades for client application. The datastore 20 is linked 
to the server 18, and is managed so as to enable the automatic provisioning of security 
relationships with nodes 12 in a network. A network having secure communication between 
these nodes 12 is typically known as and fiom herein refrared to "a virtual private network" 
(VPN). The centrally managed system 10 allows for arbitraiy additions^ modifications, and 
1 0 alterations to the datastore 20 and, in turn, deploys that information througfh the server 1 STto 
nodes 12 located vrithin a virtual private network. 

The method of establishing secure communication between nodes in a work group is detailed 
in Figure 2. On startup of a node within a work groiqp, the client application 14 instructs the 
15 node 12 to form a connection with the server 18. Once the instructions have been received, as 

indicated at 102, a socket connection is formed between &at same node 12 and server 1 8 

•J* 

(generally using secure socket links such as SSL/3DES socket security). Once the 
connection, 104, is formed between the server and the node, the authentication phase, 106, 
begins. The cUent appUcation transmits credentials to the server 18.^The server 18 then 

20 authenticates the validity of these credentials and returns data stating the success 108 or 
failure 109 of the logon to the server. If the credentials are found to be invalid the process 
fails and ends. Once the node is logged onto the server 18 and a secure connection is formed, 
the synchronization phase 110 begins. The server 18 deUvers a packet of configurational 
information to the cUent appUcation 14 of a node 12 via the secure socket connection so as to 

25 establish a virtual private network. The configurational information includes, but is not 
limited to, a list of virtual private networks to which that node is a member, their related 
attributes, the state of other nodes located within a VPN of which the node or cUent computer 
is a member, and their related details such as IP address. Once this transfer of information 
1 12 has occurred, the server 18 and node 12 are successfully linked as indicated at 114, and 

30 the abiUty to transfer data over a secure line of communication is enabled. Once a node is 
logged onto the server 18, data is transferred between a pair of nodes 12 by invoking 
procedures on remotely hosted applications on ttie node 12 and determining the type and 
target of the change or data to be distributed. 
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Tlie system 10 is global by nature such that it facilitates the central management of the VPN. 
The system 10 enables each node 12 and server 18 to be informed of any change to the VPN 
by updating a single point within the VPN and transmitting that data to all afifected members 
5 of the VPN. Once a node is logged on to a VPN, thereafter, any change to the datastore 20 
that affects a work group of which the node 12 is a member will be forwarded from the server 
18 to that node. The server is able to determine the relevant nodes 12 from the contents of 
the data product received during the information transfer phase 1 12. There are two types of 
changes that affect the datastore 20. A node generated change e.g7 going ofEIine, invokes an 
1 0 implication located on the server 18 to change the attribute of **itself *. The server 18 

examines ttie type of change, in fhis case - going ofiQine, and determines all online nodes in 
the VPN's that the node is a member of which require notification. The server 18 retrieves a 
list of those nodes from the datastore 20, and notifies each mterested node. The notification 
is either synchronous or asynchronous. 

15 

A management interfece change e.g7 altering VPN membership for example, through a web- 
based configuration tool, invokes a procedure on the server 18 notifying the server 18 of the 
change to the datastore 20. The server 18 examines the type of change and distributes the 
notification as described above. Accordingly, a VPN is established to allow communication 
20 between each of the nodes. A similar procedure may be utiUzed in the configuration of 
Figure 3. 

Figure 3 illustrates a plurality of nodes 12A through 12E, where at nodes 12C through 12E 
there are a plurality of client computers. The computer system 10 detailed in Figure 3 is a 

25 multi-tiered client/server system in which every node 12 acts as both a cHmt and server. A 
node either pulls update from the server, and in such a case in synchronous or acts as a client, 
or the server pushes i]^dates to a node by invoking a method on an object which resides on 
the node, hence is asynchronous and acts as a server. The server 18 op^tes over an existing 
network connection to fiie Intemet 22 that each node 12 possesses. The computer system 10 

30 allows arbitrary groi5)ing of nodes 12 on flie Intemet 22 into VPNs across, for instance, 
network, organisational and geographical boundaries. 
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The computer system 10 enables an extranet connection for example between two offices of a 
company 12D and 12E, each of which includes its own Intranet, to be included in a work 
group. In this situation a corporation typicaUy will have at least one localized server 17B, 
19B, which will act as server for that Intranet Each node 12 within that corporation will be 

5 connected to that localized server. The localized server 17B, 19B exists within a hierarchy 
within the computer system such that if a node/client computer within the corporation queries 
the localized server, and that servw does not contain the information queried for, that server 
climbs ttie hierarchy chain to a higher vp server and queries for the information. This process 
continues until the information is returned to the localized server where it can be distributed 

10 to the appropriate client computers within that network. Alternatively, a node within the*** 
corporate network is capable of conmiunicating with, for example a traveling user 12B 
located outside the office. 

When each node 12A through 12E logs onto flie server 18, such that each node in the network 
15 exists in a parallel relationship with another node. In one embodiment, each pair of nodes is 
typically setup with a set of keys and a unique identity such that they may transmit secure 
messages that have been encrypted and decrypted using this set of pair based keys. 
Preferably, the system 10 employs an existing peer-to-peer key exchange mechanism e.g. 
Internet Key Exchange (IKE), to negotiate session keys with each peer for data exchange. 
20 However, in the event that IKE is inaccessible, a pair of nodes 12 may negotiate and transmit 
keys via server 18, In the alternative, the server 18 may generate and distribute to keys and 
node pairs 12. It will be appreciated that when transmitting data between two nodes logged 
on to a virtual private network, that data is not transmitted through the server 18. The server 
18 is used for the initial provisioning of the virtual private network and to transfer 
25 information to the client applicationl4 of each node 12 with configuration information for the 
provisioning of that virtual private network. Again a VPN is established between a set of 
nodes interconnected by the Intemet 22. 

Figure 4 again shows computer system 10, and in this embodiment, involves &e use of a 
gateway 24 that includes a library portion containing attributes of the servers coimected to the 
30 gateway 24. Although the gateway 24 controls access to several nodes, each indicated as a 
server 25, the gateway 24 is considered a node by other users within the VPN and typically 
includes a key pair associating it with each of the other nodes in the system 10. During the 
logon process detailed in Figure 2, the server 18 vnll detect the presence of the gateway 24 
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and, during the synchronization phase, the datastore 20 will provide infoimation to the 
gateway 24 as to the range of IP addresses that are assigned to nodes behind the gateway. In 
an alternative embodiment, the. server will also detect the presence of a firewall 23 (shown in 
Figure 4), NAT box, or PAT box (not shown) as above. The gateway 24 includes a set of 
5 rules called security associations that are designed to control access to the VPN such that the 
gateway protects a plurality of nodes. Conventionally, when a node in firont of the gateway, 
such as 12A wishes to communicate with a node behind the gateway such as 12G, the node 
12A selects the key pair associated with the gateway 24 to provide encryption and decryption 
of the data. The decryption then occurs at the gateway as opposed to at the node to which the 

1 0 message is directed. The same is true of a NAT device where decryption traditionally occurs 
at the device. When a user who is typically a member of the plurality of nodes located 
behind the gateway, such as a company networic 12G, is working from home 12A, the TP 
address of the home computer 12 A is not in the range of P addresses specified by the 
gateway 24. When an IP address fells outside the range of addresses known to the gateway 

15 24 access may be denied to the company netwo±. In such a situation, a virtual IP (VIP) 
address is typically assigned to the home user 12A. When a VIP is assigned to the node of 
the home user 12A, data sent firom node 12A to the company network 12G, located behind 
the gateway 24, the gateway will route this data through a virtual interface. In the case where 
a node is a intranet, as in Figure 3 node 12C, and that node 12C wants to send data to 19B, 

20 the server 1 8 will have a plurality of rules known as an access control list (ACL), stating 
which chent computers located within 12C may access data on the servers. Security 
measures in each of the above cases conventionally are employed at the gateway 24. 

In order to employ end to end security in the presence of firewalls, gateways, NAT/PAT 
25 boxes, and proxy servers or when connections are slow and imreliable, a pref^ed procedure 
is set forth in Figure 5 is utilized. On startup of a node 12 within a work group (as shown in 
Figures 3 and 4), that node fora:is a secure connection with server 18, as described in Figure 
2. Once connected to the server, 202, on synchronization a mechanism assesses connectivity 
between nodes and determines the presence of NAT devices, firewalls, gateways and proxy 
30 servers in front of particular nodes withm the VPN. On assessing connectivity, 204, where a ' 
node is located behind for example, a NAT or PAT box, that configurational information is 
conveyed to the client application of each member within the VPN. Provided a node is not 
located behind a gateway, NAT/PAT box, firewall, or proxy server, a data packet, originating 
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from independent applications, is sent securely from one node 12 to another typically 
employing conventional methods of end-to-end secmity. Such packets typically comprise an 
IP header 72, a TCP header 74, and data 76 as shown in Figure 7a. The IP header 
communicates the data endpoint, the TCP header specifies the transport protocol, and the data 
portion is the bit stream which comprises the message bemg sent. The actual processing of 
the infonnation contained within the data packets, as well as the decryption, is known in the 
art and &lls outside the scope of this invention. 

In the event that a device is detected in front of a particular node, the system 10 enq}loys a 
modified method of communication that &cilitates end-to-end security and is described 
below. The detection of a NAT device, firewall, gateway, and proxy server, 206, indicates to 
the system 10 to invoke a modification to the data packet in order to facilitate traversing of 
tiie device. Data packets, originating &om a node within the VPN are intercepted, 207 and 
those packets destined to a specific VPN node located behind a device are selected for further 
processing. The selection for further processing informs the system 10 that these data 
packets that have been intercepted require modification in order to, enable theur sending. 
Thus, the data packets are examined and packet headers are modified 208 (as shown in Figure 
7) as will be described below. This masques the data packets such that, to the device they 
appear to be immodified and traverse the device as secure encrypted data packets. The 
masqueraded data packets preserve the original data packet and header infonnation as an 
encapsxilated secure payload and appends a new external header. The external header 
includes a data bit from herein referred to as a **masquerade bit" which acts as a "flag" or 
"indicatof that the packet header has been modified, 210. To the device, such as those shown 
in Figures 3 and 4, the data packet appears to be an uimiodified protocol session and passes 
througih file device unread. In the case of a firewall, (shown in Figure 4) upon receipt at the 
firewall, the external header is identified as an SSL and is directed to dedicated port 443 m 
the wall and passes through that port without fiirOier examination to the intended receiver. 

In the preferred embodiment, the system nodes are restricted to use Encapsulated Security 
Payload (ESP) protocol in tunneling for securing data being exchanged by VPN nodes. This 
is a protocol that resides on top of the IP layer in network stack and thus allows for securing 
any IP traffic. A data packet secured by Tunneled ESP is encrypted as a whole, and is 
prepended with an ESP header and another copy of IP header which comprises a new 
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external header. Source/destinatioii node infonnation in the new JP header within the external 
header may differ from the IP header in original data packet. The ESP processing setup 
determines any change to the ff header information. Original IP header is further referred as 
'intemar and newly prepended one - as 'external'. 

5 

Typically, when an encrypted packet traverses a NAT device, for example, its external IP 
header is modified to contain proper addressmg information. Upon arrival at flie destination 
node the external IP header is stripped off during data processing and the external JP 
addressing information is irrevocably lost Therefore, ttie receiving node is not able to 

10 process the decrypted packet properly. In the present invention, the data packet memorizing 
the external IP header prior to its stripping, and ften adjusts internal IP header based on the 
network setup. For example, a data packet when traversing a NAT device, arrives at the NAT 
device and at this point pronq)ts the system to copy the destination IP address from the 
external header. I^ in addition, the data packet arrives from a NAT'ed node (a node having a 

15 NAT device in front), then the system is further prompted to update flie source IP address 
from the external header. The IP/TCP/UDP checksums of the adjusted packet are recalculated 
or turned off such tiiat the packet integrity is guaranteed by successful decryption. The 
centralized nature of the VPN supplies nodes with infonnation about their peers that allows 
for each node to decide if a particular peer or node is NAT'ed. This effectively eliminates the 

20 ^detection' (or 'negotiation') step known by those skilled in the art and typically employed by 
other NAT-traversal methods to determine the presence of the NAT between two nodes. The 
process described above of changing the IP header before submitting a data packet to the IP 
processing is further referred to as 'RNAT transformation'. 

25 A data packet traversing a PAT has both its IP head^ modified as well as its transport layer 
header translated. Commonly siq>ported transport protocols are TCP and UDP. ICMP, while 
not being true transport protocol, is also generally provided a limited support for its ECHO 
messages. Note that these fliree protocols are referred as 'post-IP protocols' below. 

30 In the case where a data packet traverses a PAT device, the system employs the following 
approach. Assmne node A being PAT'ed node (a node having a PAT device located in front) 
and node B its peer residing outside the PAT device. In this case, node B may be located 
behind NAT, but not PAT device. A packet sent by node A is processed as described and 
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above and then in turn, receives a UDP header and a masquerade bit inserted between IP and 
ESP headers of the encrypted packet as was described above. This extra step of outbound 
processing, including the UDP header, is fiirther referred as 'UDP-masquerading' or 
'masquerading'. The masquerade allows recipiait to differentiate between masqueraded and 

5 *true' UDP packets with a high degree of accuracy. Upon arrival of a data packet at node B 
having traversed a PAT device, the data packet UDP header is associated with the tunnel 
through which it arrived. In oth^ words, it associates the node from which the data packet 
originated. Then packet is then stripped of the UDP masquerade header to reveal the original 
header and inbound ESP processing and RNAT transformation is performed as previously 

10 outlined. The ESP code links plam text post-IP information to the tunnel through which it 
was delivered. 

A data packet leaving node B destined for node A is first subject to a regular ESP processing 
with compulsory Tunnel selection based on its IP and post-IP information stored during 
15 inbound processing. Once encryption of the data packet is completed, the data packet is 
masqueraded based on masquerading information also stored during inbound processing. 
Upon arrival at node A, the data packet is subject to demasquerading, regular ESP processing 
and RNAT transformation. 

20 In a further embodiment, the system facilitates a means to potential post-IP information 
ambiguity developing on node B after packet decryption. For example, two nodes (Al, A2) 
may reside behind the same PAT device and use the same source port to access the same 
node B port. It this case, after KNAT is applied, data packets originating from nodes Al and 
A2 are indistinguishable and a reply from node B could not be routed back to the appropriate 

25 node. The system in this case applies a post-IP layer overloading (similar to the PAT) to each 
data packet traversing the same PAT device arriving through different tunnels. A PAT 
transformation is applied to all inbound data packets to resolve ambiguities and the reverse 
mapping to the originating node is performed on the outbound data packet in order to restore 
the post-IP headers to peer's expectations. 

30 

When a node is the intended recipient and that node logs on to the VPN, the node receives a 
data packet 252 as shown in Figure 6. When a data packet arrives, the interception 
. mechanism (253) analyses the packet header 254 for the presence of a masquerade bit. If a 
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masquerade bit is not detected, the data packet is received by the intended node 262 and is 
processed When a masquerade bit is detected 256, it indicates to the system that further 
processing is required. When the received node is located behind a NAT/PAT box, it is the 
box that receives the data packet, analyzes the header, and detects the presence of a 
5 masquerade bit In the case where there is no NAT/PAT box, the node performs the analysis 
and detects the masquerade bit Once the masquerade bit is found, the extemal header is 
removed 258 to reveal to origLoal header. This original header is examined and the packet is 
routed to the intended-receiving node and allows for return data to be sent 

10 If, in the above circumstance, the node is not logged on to a VPN, the packet is sent and once 
&e peer or intended receiving node logs on to a VPN the packet is received by the peer 
following the procedure outlined in Figure 6. 

Figure 7 shows the transformation of a regular data packet 70 illustrated in Figure 7a to a 
IS modified data packet 90 illustrated in Figure 7b that was described m Figure 7. The 

originating data packet 70 includes an IP header 72, a TCP header 74, and a data portion 76. 
In order to facilitate end-to-end security in the presence of a firewall, NAT/PAT box or 
gateway etc, the data packet is modified/re-written, as described in Figures 5 and 6. The 
modified data packet 90 comprises a new header 91and a data payload 96. The header 91 of 
20 the modified packet 90 comprises an IP header 72b, and ESP header 93 and a masquerade bit 
94. The data payload 96 of the modified pack 90 encapsulates the original data packet 70. On 
receiving a modified packet, as detailed in Figure 6, the new header 91 is removed and the 
packet is processed to reveal the original data packet 70. 

25 On securing a communications path over a pubUc network between two nodes in a computer 
work group, a typical encryption technique used to transfer data between these nodes 
includes: generating a data packet to be transmitted over the seciured communications path 
wh^e the data packet includes routing information; encrypting that data packet using an 
encryption technique known to one skilled in the art; oicapsulating the encrypted data packet 

30 into a secondary data packet compatible with public network protocols; transmitting the 
encapsulated data packet over the public network; the data packet arriving at the receiving 
node; and tiiat receiving node unpacking the encrypted data packet using a set of 
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authentication keys, stripping the second data packet from the original data packet, and 
decrypting that data packet received from the originating node. 

In the preferred embodiment, secure ff communication using end-to-end security between any 
5 two nodes 12 over the Internet 22 is established with only minimal assumptions about any 
particular node's connectivity privileges. This is accomplished by ^plying EPSec 
transformations to inconodng and outgoing IP packets at the transport layer and then 
transfoiming these processed packets so they appear to be an SSL protocol session until 
received by the destination node. 

10 

For operation within the system, the node (base configuration) preferably mcludes: 

• An IP address and a connection to the Internet (may be non-unique); and 

• Ability to send and receive TCP data on port 443 in SSL format (on some servers 
may also require the ability to send and receive TCP data in SSL format on a port 

1 5 specified by the server). 

The optimal configuration for a node (recommended configuration)^is defined as follows: 

• Those abilities defined in the base configuration; and 

• A globally routable IP address or 1 : 1 static NAT. 

20 At least one node in each pair supports at least the recommended configuration, and the other 
node supports at least the minimum configuratioa The system requires that only one of a pair 
of nodes may be located behind a firewall. The recommended encryption level for data in 
transit is 3DES. The system, in the preferred embodiment, accesses both: 

• configuration data (BP addresses, etc) provided by server, client application, and 
25 library aforementioned; and 

• a packet interception and injection mechanism partially provided by Trilogy 
AdntiitOne 

The compute system 10 may be run on adiverse set of operating systems and hardware 
30 platforms such as open BSD, UNIX, Windows NT, Windows 95/98, Linux, and Solaris. 

In another embodiment, as shown in Figure 8, a system 50 comprises VPN servers 44, which 
function as central policy management for estabUshing and facilitating VPN operation. The 
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system 50 further comprises at least a pair of database servers 40 and a Round-Robin Domain 
Name Server (DNS) 42 in a distributed, fully integrated environment. The DNS server 42 
assures homogenous distribution of the data load across the VPN servers 44. Connectivity 
between VPN servers 44 and the database servers 40 is implemented so as to support several 
5 modes of communication including but not limited to open database connectivity (ODBC), 
Java Database Connectivity (JDBC) or any other database connectivity interface. The 
database servers 40 are mutually synchronized to keep the data contents current and up-to- 
date. The content of each database server 40 is identical such tiiat, should one database 
server 40 crash, each of the VPN servers 44 connected to that £dled database server 40 may 
1 0 automatically reconnect to another available non-£uled database server. 

The VPN server 44 may operate in either a standalone or a distributed environment. The 
nodes 12 participating in a VPN may be connected to the same VPN server 44, as the VPN 
servers 44 are synchronized such that a node may log onto any VPN server 44 and participate 
15 in a VPN of which they are a member. As the system 50 is fully synchronized, forwarding 
from one VPN server 44 to another is not necessary. Each event or revised attribute of a 
node 12 or server 44 is distributed to the entire system 50 directly by the original sender. 
. Synchronization enables VPN nodes to see one another as if they wei:e physically connected 
to the same VPN server 44. 

20 

The system 50 employs a variety of communication protocols utilized within the VPN 
environment so as to fecilitate communication of the VPN server 44 and its node 12 across 
the open network environment In the preferred embodiment, commxmication within the 
system 50 occurs at a "secure sockets layer*' (SSL) underneath any security attributes. The 
25 system however, further enables commxmication, in one embodiment at the application layer. 
Such communication may be in the form of the following: 

a) Authentication of users 

When a VPN node 12 is going online, the node 12 submits its authentication credentials, 
30 which are validated on the server side. The node 12 may enter another state of 

communication once the authentication credentials have been approved. The system 50 
supports two ways of authentication, either using a user name and password or client side 
certificates however, authentication is not limited to these two types. 
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b) Proxy authentication of users 

On authenticating the credentials of a node 12, the credential(s) is validated against an 
external data repository, for example Lightweight Directory Access Protocol (LDAPO, 
5 Radius, or Windows NT/2000 domain. 

c) Distribution of user state updates 

When a VPN node 12 goes online/offline, other nodes within the VPN are notified of this 
update such that the related security associations are also updated. Any further 
1 0 communication between VPN nodes is utilized through an IPSec protocol and does not flow 
flirough the VPN server 44. 

d) Providing a way to establish conmion secret 

Each VPN node 12 generally possesses a common secret such as a private key which is 
IS passed to the IPSec layer and is used to protect the respective data traffic. This secret may be 
created by the VPN server 44 and distributed to the appropriate VPN node or the secret may 
be created locally at the node 12 and submitted to a second node in a secure and private 
manner through the VPN server 44. The common secret for example may be a symmetric 
key, "Internet key exchange" (KE) so as to 2dlow secured node-to-nbde communication. 
20 e) Password exchange protocol 

The system 50 encapsulates a secure-transaction mechanism to allow VPN nodes 12 to 
update their VPN passwords. After anode is successfully authenticated, die node is allowed 
to submit a password change request, followed by the approval/confirmation of both 
communication parties (VPN node and VPN server 44). 

25 

Aldioug^ the invention has been described with refCTence to certain specific embodiments, 
various modifications thereof will be apparent to those skilled in the art without departing 
firom the spirit and scope of tiie invention as outlined in the claims appended hereto. 
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CLAIMS: 

1 . A method for establishing a system for secure communications between nodes in a 
workgroup over a public network by facilitating the creation of a virtual private network 

5 (VPN), including a VPN server, tiie method con5)rising the steps of: 

esta b lishing a secure connection between at least a pair of nodes within said workgroup 
and said VPN server; and 

synchronizing each of said connected nodes with said VPN s^er such that each of said 
1 0 comiected nodes receives configurational information relating to attributes of each of sai3* 
other coimected nodes; 

wherein, whra an attribute relating to one of said comiected nodes or said VPN server is 
revised, said configurational information relating to said attribute is updated at each of said 
comiected nodes. 

15 

2. The method for establishing the system of claim 1, further comtprising, following said 
step of 

establishing said secure connection, a sX&p of authorizing, at said VPN server, validity of said 
connection between said VPN server and each of said connected nodes. 

20 

3. The method for establishing the system of claim 1, wherein following said step of 
synchronizing said server and each of said connected nodes, a step of sensing attribute 
revisions relating to one of said connected nodes or said server. 

25 4. The method for establishing the system of claim 1, wherein said VPN server enables 
secure 

exchange of said configurational information between said connected nodes. 

5. The method for establishing the system of claim 1, wherein said VPN server restricts 
30 exchanges of configurational information based on trust relationships established by said 
connected nodes. 



6. 



The method for establishing the system of claim 1, wherein each of said connected nodes 
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remains in a loop with said VPN server so as to forward any attribute revisions changes 
within a node to each of said connected nodes. 

7. The method for establishing the system of claim 1, wherein each of said connected nodes 
5 automatically pull changes firom said VPN server so as to update said configurational 

information stored at said node. 

8. A system for establishing secure communication between nodes in a workgroup over a 
public 

10 network by facilitating the creation of a virtual private network, tiie system comprising: 
at least a pair of nodes; 

a VPN server, connected with each of said at least a pair of nodes for synchronizing each of 
said connected nodes with said VPN server such that each of said connected nodes receives 
15 configurational information relating to attributes of said other connected nodes or said VPN 
server, 

wherein, when an attribute relating to one of said connected nodes or said server is revised, 
said configurational information relating to said attribute is updated at each of said connected 
nodes. 

20 

9. The system of claim 8, wherein said system further comprises a datastore connected to 
said 

server. 

25 10. The system of claim 8, wherein said system further comprises a client application located 
at 

each of said connected nodes. 

1 1 . A method for establishing a system for secure transfer of a data packet between a first 
30 node 

and a second node in a workgroup over a public network, where said nodes are members of a 
virtual private network, the method comprising the steps of: 
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assessing a presence of a device associated with said connected first and second nodes; 
modifying a packet header of said data packet intended for transfer between said first and 
second nodes when a device is detected; 

wherein said modification of said packet headers facilitates traversing said detected device 
5 for transmission of said data packet between said first node and said second node. 

12. The method for establishing the system of claim 11, wherein said modified packet header 
comprises an Encq)sulated Security Payload (ESP) header, an Internet Protocol (IP) header, 
and a masquerade bit, said masquerade bit acting as an indicator to one of said first and 

1 0 second nodes that said data packet has been modified. 

13. The method for establishing the system of claim 12, wherein said masquerade bit is 
located 

between said ESP header and said IP header. 

15 

14. The method for establishing the system of claim 12, wherein a packet interception 
mechanism analyses said packet headers for detecting the presence of said masquerade bit. 

15. The method for establishing the system of claim 13, wherein when said masquerade bit is 
20 detected within said packet header, said modified packet header is removed and the original 

packet header of said data packet routes said data packet to one of said first and second node. 

16. The method for establishing the system of claim 11, wherein said device is selected from 
a 

25 group comprising a Network Address Translation (NAT) Device, a firewall, a gateway, a 
proxy server, and combinations thereof. 

17. The method for establishing the system of claim 11, wherein when a device is detected, 
said 

30 device is located in front of said node. 

18. A computer system for establishing the secure transfer of a data packet between nodes in 
a 
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workgroup over a public netwoilc, where said nodes are members of a VPN, the system 
comprising: 

a first node; 
5 a second node; 

a device detection mechanism; and 
a packet interception mechanism; 

wherein when a data packet is transferred from said first node to said second node and a 
device is detected at said second node, said data packet is intercepted and a packet header of 
1 0 said data packet is modified to &cilitate the data transfer betiveen said nodes. 
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